Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*: support canceling DDL statements with KILL #35803

Merged
merged 7 commits into from
Jun 29, 2022
Merged

*: support canceling DDL statements with KILL #35803

merged 7 commits into from
Jun 29, 2022

Conversation

morgo
Copy link
Contributor

@morgo morgo commented Jun 28, 2022

What problem does this PR solve?

Issue Number: close #24144

Problem Summary:

In v6.1+ the KILL statement now works as expected because the global kill feature is enabled by default. However, this doesn't apply to DDL statements which are uninterruptible and instead require ADMIN CANCEL DDL.

Requiring a different statement for DDL is not MySQL compatible, and difficult to expect of new users to understand that they must use a tidb-specific statement.

What is changed and how it works?

The KILL statement can now be used to terminate running DDL statements.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

I found this difficult to automate without adding sleep calls, but I might be missing something. Here is the manual test I used:

# session1
mysql> select count(*) from t1;
+----------+
| count(*) |
+----------+
|  4851010 |
+----------+
1 row in set (2.65 sec)

mysql> alter table t1 add index(a);
ERROR 8214 (HY000): Cancelled DDL job

# while alter table is running in session1
mysql> admin show ddl jobs; show processlist; kill 2199023255957; select sleep(1);admin show ddl jobs; show processlist;
ERROR 2013 (HY000): Lost connection to MySQL server during query
No connection. Trying to reconnect...
Connection id:    407
Current database: *** NONE ***

+--------+---------+----------------------------------+--------------+----------------------+-----------+----------+-----------+---------------------+---------------------+---------------------+---------+
| JOB_ID | DB_NAME | TABLE_NAME                       | JOB_TYPE     | SCHEMA_STATE         | SCHEMA_ID | TABLE_ID | ROW_COUNT | CREATE_TIME         | START_TIME          | END_TIME            | STATE   |
+--------+---------+----------------------------------+--------------+----------------------+-----------+----------+-----------+---------------------+---------------------+---------------------+---------+
|     71 | test    | t1                               | add index    | write reorganization |         1 |       69 |         0 | 2022-06-28 11:13:00 | 2022-06-28 11:13:00 | NULL                | running |
|     70 | test    | t1                               | create table | public               |         1 |       69 |         0 | 2022-06-28 10:56:03 | 2022-06-28 10:56:03 | 2022-06-28 10:56:03 | synced  |
|     68 | mysql   | advisory_locks                   | create table | public               |         3 |       67 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     66 | mysql   | analyze_jobs                     | create table | public               |         3 |       65 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     64 | mysql   | stats_meta_history               | create table | public               |         3 |       63 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     62 | mysql   | stats_history                    | create table | public               |         3 |       61 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     60 | mysql   | analyze_options                  | create table | public               |         3 |       59 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     58 | mysql   | table_cache_meta                 | create table | public               |         3 |       57 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     56 | mysql   | column_stats_usage               | create table | public               |         3 |       55 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     54 | mysql   | capture_plan_baselines_blacklist | create table | public               |         3 |       53 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
|     52 | mysql   | global_grants                    | create table | public               |         3 |       51 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced  |
+--------+---------+----------------------------------+--------------+----------------------+-----------+----------+-----------+---------------------+---------------------+---------------------+---------+
11 rows in set (0.00 sec)

+---------------+------+-----------------+------+---------+------+------------+-----------------------------+
| Id            | User | Host            | db   | Command | Time | State      | Info                        |
+---------------+------+-----------------+------+---------+------+------------+-----------------------------+
| 2199023255957 | root | 127.0.0.1:64450 | test | Query   |    3 | autocommit | alter table t1 add index(a) |
| 2199023255959 | root | 127.0.0.1:52377 | NULL | Query   |    0 | autocommit | show processlist            |
+---------------+------+-----------------+------+---------+------+------------+-----------------------------+
2 rows in set (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

+----------+
| sleep(1) |
+----------+
|        0 |
+----------+
1 row in set (1.01 sec)

+--------+---------+----------------------------------+--------------+----------------------+-----------+----------+-----------+---------------------+---------------------+---------------------+------------+
| JOB_ID | DB_NAME | TABLE_NAME                       | JOB_TYPE     | SCHEMA_STATE         | SCHEMA_ID | TABLE_ID | ROW_COUNT | CREATE_TIME         | START_TIME          | END_TIME            | STATE      |
+--------+---------+----------------------------------+--------------+----------------------+-----------+----------+-----------+---------------------+---------------------+---------------------+------------+
|     71 | test    | t1                               | add index    | write reorganization |         1 |       69 |         0 | 2022-06-28 11:13:00 | 2022-06-28 11:13:00 | NULL                | cancelling |
|     70 | test    | t1                               | create table | public               |         1 |       69 |         0 | 2022-06-28 10:56:03 | 2022-06-28 10:56:03 | 2022-06-28 10:56:03 | synced     |
|     68 | mysql   | advisory_locks                   | create table | public               |         3 |       67 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     66 | mysql   | analyze_jobs                     | create table | public               |         3 |       65 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     64 | mysql   | stats_meta_history               | create table | public               |         3 |       63 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     62 | mysql   | stats_history                    | create table | public               |         3 |       61 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     60 | mysql   | analyze_options                  | create table | public               |         3 |       59 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     58 | mysql   | table_cache_meta                 | create table | public               |         3 |       57 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     56 | mysql   | column_stats_usage               | create table | public               |         3 |       55 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     54 | mysql   | capture_plan_baselines_blacklist | create table | public               |         3 |       53 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
|     52 | mysql   | global_grants                    | create table | public               |         3 |       51 |         0 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | 2022-06-28 10:55:30 | synced     |
+--------+---------+----------------------------------+--------------+----------------------+-----------+----------+-----------+---------------------+---------------------+---------------------+------------+
11 rows in set (0.00 sec)

+---------------+------+-----------------+------+---------+------+------------+-----------------------------+
| Id            | User | Host            | db   | Command | Time | State      | Info                        |
+---------------+------+-----------------+------+---------+------+------------+-----------------------------+
| 2199023255959 | root | 127.0.0.1:52377 | NULL | Query   |    0 | autocommit | show processlist            |
| 2199023255957 | root | 127.0.0.1:64450 | test | Query   |    4 | autocommit | alter table t1 add index(a) |
+---------------+------+-----------------+------+---------+------+------------+-----------------------------+
2 rows in set (0.00 sec)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

The `KILL` statement can now be used to terminate running DDL statements.

morgo added 3 commits June 19, 2022 22:05
@morgo
wip
7981304
@morgo
Merge remote-tracking branch 'upstream/master' into kill-ddl
afde8a8 
* upstream/master:
  sessionctx: support encoding and decoding session variables (pingcap#35531)
  planner: add batch_point_get access object (pingcap#35230)
  sessionctx: set skipInit false for TiDBOptProjectionPushDown and TiDBOptAggPushDown (pingcap#35491)
  *: add support for disabling noop variables (pingcap#35496)
  lightning: add store write limiter (pingcap#35193)
  expression: avoid padding 0 when implicitly cast to binary (pingcap#35053)
  types: fix creating partition tables fail in ANSI_QUOTES mode (pingcap#35379)
  executor: add the missed runtime stats when the index merge partial task returns 0 row (pingcap#35297)
  statistics: batch insert topn and bucket when saving table stats (pingcap#35326)
  parser: Add support for INTERVAL expr unit + expr (pingcap#30253) (pingcap#35390)
  config: add missing nodelay example (pingcap#35255)
  *: Introduce `OptimisticTxnContextProvider` for optimistic txn (pingcap#35131)
  statistics: fix panic when using wrong globalStats.Indices key (pingcap#35424)
  *: fix store token is up to the limit in test (pingcap#35374)
  *: enable more flaky and update bazel config (pingcap#35500)
  ddl: expose getPrimaryKey() as a public method of model.TableInfo (pingcap#35512)
  expression, util: add `KeyWithoutTrimRightSpace` for collator (pingcap#35475)
  infoschema: try on each PD member until one succeeds or all fail (pingcap#35285)
@morgo
Merge remote-tracking branch 'upstream/master' into kill-ddl
11e57df 
* upstream/master: (57 commits)
  types: fix incompatible implementation of jsonpath extraction (pingcap#35320)
  planner: fix TRACE PLAN TARGET = 'estimation' panic when meeting partition table (pingcap#35743)
  *: Add `testfork.RunTest` to run multiple tests in one function (pingcap#35746)
  sessionctx/variable: add tests to ensure skipInit can be removed (pingcap#35703)
  helper: request another PD if one of them is unavailable (pingcap#35750)
  metrics: add cached table related metrics to grafana panel (pingcap#34718)
  expression: use cloned RetType at `evaluateExprWithNull` when it may be changed. (pingcap#35759)
  executor: fix left join on partition table generate invalid lock key (pingcap#35732)
  readme: remove adopters (pingcap/docs#8725) (pingcap#35124)
  *: only add default value for final aggregation to fix the aggregate push down (partition) union case (pingcap#35443)
  planner: fix the wrong cost formula of MPPExchanger on cost model ver2 (pingcap#35718)
  sessionctx: support encoding and decoding statement context (pingcap#35688)
  txn: refactor ts acquisition within build and execute phases (pingcap#35376)
  ddl: for schema-level DDL method parameter is now XXXStmt (pingcap#35722)
  *: enable gofmt (pingcap#35721)
  planner: disable collate clause support for enum or set column (pingcap#35684)
  *: Provide a util to "pause" session in uint test (pingcap#35529)
  ddl: implement the core for multi-schema change (pingcap#35429)
  parser: XXXDatabaseStmt now use CIStr for DB name (pingcap#35668)
  *: remove real tikv test on github actions (pingcap#35710)
  ...
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jun 28, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • tangenta
  • xhebox

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Jun 28, 2022
@morgo morgo requested a review from zimulala June 28, 2022 17:28
@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 28, 2022
@sre-bot
Copy link
Contributor

sre-bot commented Jun 28, 2022 •
edited
Loading

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/5460a00be47473d0f97551418c8bfc93ca9056f2

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 29, 2022
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 29, 2022
@xhebox
Copy link
Contributor

xhebox commented Jun 29, 2022

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 14ce59e

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 29, 2022
@ti-chi-bot ti-chi-bot merged commit 41c0ab4 into pingcap:master Jun 29, 2022
@sre-bot
Copy link
Contributor

sre-bot commented Jun 29, 2022

TiDB MergeCI notify

🔴 Bad News! [1] CI still failing after this pr merged.
These failed integration tests don't seem to be introduced by the current PR.

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-common-test 🔴 failed 2, success 9, total 11 18 min Existing failure
idc-jenkins-ci/integration-cdc-test 🟢 all 35 tests passed 26 min Existing passed
idc-jenkins-ci-tidb/common-test 🟢 all 12 tests passed 12 min Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1 🟢 all 26 tests passed 6 min 42 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 6 min 3 sec Existing passed
idc-jenkins-ci-tidb/integration-ddl-test 🟢 all 6 tests passed 5 min 53 sec Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 5 min 6 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 2 min 54 sec Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test 🟢 all 1 tests passed 2 min 34 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed

morgo added a commit to morgo/tidb that referenced this pull request Jun 29, 2022
@morgo
Merge remote-tracking branch 'upstream/master' into remove-hidden-sys…
a104dff 
…vars

* upstream/master: (28 commits)
  *: enable unconvert (pingcap#35821)
  log-backup: check timezone when using datatime format ts string (pingcap#35811)
  planner: support the leading hint for outer join (pingcap#35264)
  lightning: revert new policy of allocating rowid and refine it later (pingcap#35817)
  txn: manage the initialization of RCCheckTS by transaction context provider (pingcap#35554)
  ddl: make output field name in `show tables/databases stmt` compatible with mysql (pingcap#35136)
  txn: remove `legacy.SimpleTxnContextProvider` (pingcap#35667)
  txn: fix the error message if retry could not be used for rc check ts (pingcap#35474)
  *: support canceling DDL statements with KILL (pingcap#35803)
  lightning: avoid extracting db schema when schema file is not provided (pingcap#35783)
  *: enable all gosimples (pingcap#35762)
  tools/check: skip 'tests' directory in 'make ut' (pingcap#35800)
  metrics: add loading region cache duration (pingcap#34679)
  ddl: DROP TABLE/VIEW/SEQUENCE now use XXXStmt as parameter (pingcap#35741)
  types: fix incompatible implementation of jsonpath extraction (pingcap#35320)
  planner: fix TRACE PLAN TARGET = 'estimation' panic when meeting partition table (pingcap#35743)
  *: Add `testfork.RunTest` to run multiple tests in one function (pingcap#35746)
  sessionctx/variable: add tests to ensure skipInit can be removed (pingcap#35703)
  helper: request another PD if one of them is unavailable (pingcap#35750)
  metrics: add cached table related metrics to grafana panel (pingcap#34718)
  ...
@tiancaiamao tiancaiamao mentioned this pull request Jul 4, 2022
Merged
12 tasks
@yahonda yahonda mentioned this pull request Oct 28, 2022
Closed
@tiancaiamao tiancaiamao mentioned this pull request Mar 24, 2023
Open
This was referenced May 16, 2023
Closed
Merged
@ti-chi-bot ti-chi-bot mentioned this pull request May 17, 2023
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tangenta tangenta tangenta approved these changes

@xhebox xhebox xhebox approved these changes

@zimulala zimulala Awaiting requested review from zimulala

Assignees
No one assigned
Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

KILL TIDB or global KILL should cancel DDL
5 participants
@morgo @ti-chi-bot @sre-bot @xhebox @tangenta